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(57) Abstract 



In a method of interpolating an output color signal (Go) of a given color in dependence on an input signal (RGBin) having first 
values of the given color which are influenced by a first other color and second values of the given color which are influenced by a second 
other color, an intermediate color signal (Gc') is interpolated (1) at positions where no signal of the given color is present, an average value 
(avG) of the given color is generated (3) in dependence upon both the first and second values, and the output color signal (Go) is furnished 
(S) in dependence upon both the intermediate color signal (Gc') and die average value (avG). 



FOR THE PURPOSES OF INFORMATION ONLY 
Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 


LS 


Lesotho 


SI 


Slovenia 


AM 


Armenia 


FI 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


Luxembouig 


SN 


Senegal 


AU 


Australia 


OA 


Gabon 


LV 


Latvia 


sz 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


GN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


Burkina Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


HU 


Hungaiy 


ML 


Mali 


XT 


THnidad and Tobago 


BJ 


Benin 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BR 


Brazil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belarus 


IS 


Iceland 


MW 


Malawi 


US 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


uz 


Uzbekistan 


CF 


Central African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KE 


Kenya 


NL 


Netherlands 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Norway 


zw 


Zimbabwe 


CI 


Cdte d'lvolrc 


KP 


Democratic People's 


NZ 


New Zealand 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal 






CU 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






cz 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


LI 


Liechtenstein 


SD 


Siulan 






DK 


Denmatk 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SG 


Singapore 







wo 99/39509 



Color signal interpolation. 



1 



PCT/IB99/00073 



The invention relates to interpolation of color signals, and more 
specifically to restoration of Green uniformity of RGB Bayer sensors. 

5 RGB Bayer sensors have an alternating color pattern of rows having 

RGRG etc. colored pixels, and 
GBGB etc. colored pixels. 

In the vertical direction of a 'Red* column, a Green pixel has only Red pixels as vertically 
neighboring pixels, while in a 'Blue* column, a Green pixel has only Blue pixels as vertically 

10 neighboring pixels. Depending on the quality of the vertical color separation of the sensor, 
the Green pixels can be modulated by the amount of electron charge in the Red and Blue 
pixels. If so, then for certain colored scenes this may result in a visible column wise Green 
non-uniformity. A nice example is a highlighted Cyan color, which theoretically has no Red 
but only Blue and Green. This will result in a different Green for the Red and Blue columns, 

15 which can be visible on a display or on a printout especially because of its regularity. A 
stylistic example of the column-wise modulation of Green by the electron charges of the Red 
and Blue pixels is shown in Fig. 1. 

20 It is, infer alia, an object of the invention to reduce this Green non- 

uniformity without losing resolution. To this end, a first aspect of the invention provides a 
method as defined in claim 1 and a device as claimed in claim 3. A second aspect of the 
invention provides a camera as defined in claim 4. An advantageous embodiment is defined 
in the dependent claim 2. 

25 In a method of interpolating an output color signal of a given color in 

dependence on an input signal having first values of the given color which are influenced by 
a first other color and second values of the given color which are influenced by a second 
other color, in accordance with a primary aspect of the present invention an intermediate 
color signal is interpolated at positions where no signal of the given color is present, an 
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average value of the given color is generated in dependence upon both the first and second 
values, and the output color signal is furnished in dependence upon both the intermediate 
color signal and the average value. 

These and other aspects of the invention will be apparent from and 
5 elucidated with reference to the embodiments described hereinafter. 



In the drawings: 

Fig. 1 shows an RGB Bayer sensor in which the Green signal in the 
10 columns with Red pixels differs from the Green signal in the columns with Blue pixels; 

Fig. 2 indicates a present Green center pixel Go and four present Green 
neighboring pixels G1-G4, as well as a reconstructed Green center pixel Go' and four present 
Green neighboring pixels GI-G4; 

Fig. 3 indicates a present Green center pixel Go and two present Green 
15 neighboring pixels G1-G2, as well as a reconstructed Green center pixel Gc' and three 
present Green neighboring pixels G1-G3; and 

Fig, 4 shows a block diagram of a camera in accordance with the present 

invention. 

20 

In case of a signal processing with two row delays. Fig. 2 shows the 
declaration of the surrounding Green pixels if Green is present (left) or absent (right). In the 
latter case, the missing center Green pixel Gc' is reconstructed by means of an RGB 
reconstruction filter 1 in Fig. 4. Preferably, the RGB reconstruction filter is of the type 
25 described in EP patent application no. 97401700.6 filed on 15.07.97 (Attorneys' docket PHN 
16,466) and its corresponding applications, incorporated herein by reference. 

In practice it appears that the Green non-uniformity is limited to a certain . 
maximum, for instance 5%, of the Green signal amplitude. This Green non-uniformity level 
Th is preferably adjustable by means of the core of the camera processing. The goal of the 
30 Green uniformity restoration is that above that level Th no resolution loss will occur, but that 
below that level the center Green will be replaced by an averaged Green value of the. 
surrounding green pixels. 

The algorithm for Green uniformity restoration which yields the output 

Green value Go: 
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avG = (Gl+G2+G3-hG4)/4 

if the center Green pixel Gc' has to be interpolated then 
begin 

if abs((Gl+G2-G3-G4)/2) < Th then 
5 Go = avG 

else Go = Gc' (the reconstructed Green) 

end 

else (the center Green pixel Go is present) 
begin 

10 if abs(avG-Gc) < Th 

then Go = (Gc+avG)/2 
else Go = Gc 

end 



15 In case of a signal processing with only one row delay, only the pixels 

illustrated in Fig. 3 are available. In that case the algorithm for Green uniformity restoration 
is: 

avG = (Gl+G2)/2 

if the center Green pixel Gc' has to be interpolated then 
20 begin 

if abs(avG-G3) < Th 

then Go (avG+G3)/2 

end 

else (the center Green pixel Gc is present) 
25 begin 

if abs(avG-Gc) < Th 

then Go = (Gc+avG)/2 

else Go = Gc 

end 

30 

A block diagram of how io combine the RGB reconstruction and the 
Green uniformity restoration is shown in Fig. 4. An RGB input signal RGBin from a sensor 
S is applied to an RGB reconstruction filter 1 and to a Green uniformity restoration unit 3 is 
a present version, in a once line delayed version, and in a twice line-delayed version. The 
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RGB reconstruction filter interpolates missing red, green and blue pixels values on the basis 
of the signals applied to the filter in which, as apparent from Fig. 1, at each pixel position 
only one of the three colors R, G and B is present. A select box 5 selects between the 
original center Green pixel value Gc or a reconstructed center Green value Gc' from the 

5 RGB reconstruction filter 1 , and the averaged Green signal avG from the Green uniformity 
restoration unit 3. If the edges in the Green area are smaller than the level Th, then the 
average Green value avG is selected. If the edges in the Green area are larger than the level 
Th, i.e. when high frequencies above that level Th have been detected, then the signal Gc or 
Gc* from the RGB reconstruction filter 1 will be selected. 

10 By leaving out the upper horizontal line delay in the block diagram of 

Fig. 4. the circuit can be used for the single row RGB reconstruction and Green uniformity 
restoration on the basis of the pixels shown in Fig. 3. 

Tests with pictures of existing Bayer image sensors, having column wise 
Green non-uniformity, proved that the mentioned algorithms eliminate that non-uniformity 

15 and maintain the resolution. The estimated improvement in signal-to-noise ratio is about 2 to 
3 dB. (The estimation has been done by adding noise until both pictures give the same 
impression.) This improvement also holds for fixed pattern noise. Further this circuit can be 
applied for averaging the dark current of the image sensor. 

The following salient features of the invention are noteworthy. The 

20 restoration of the Green uniformity of RGB Bayer image sensors without resolution loss. The 
restoration with the already available row delays needed for the RGB reconstruction. The 
inherent signal-to-noise improvement and possibility to filter the dark current of the sensor. 

It should be noted that the above-mentioned embodiments illustrate rather 
than limit the invention, and that those skilled in the art will be able to design many 

25 alternative embodiments without departing from the scope of the appended claims. For 
example, instead of the average value avG, a median value or any other low-pass filtered 
green signal can be used as long as it combines Green from at least one Red column and 
Green from at least one Blue column; any such alternatives are included by the expression 
"average value" in the claims. In the claims, any reference signs placed between parentheses 

30 shall not be construed as limiting the claim. The invention can be implemented by means of 
hardware comprising several distinct elements, and by means of a suitably programmed 
computer. In the device claim enumerating several means, several of these means can be 
embodied by one and the same item of hardware. 
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Claims 



1 . A method of interpolating an output color signal (Go) of a given color in 
dependence on an input signal (RGBin) having first values of the given color which are 
influenced by a first other color and second values of the given color which are influenced by 
a second other color, the method comprising: 

5 interpolating (1) an intermediate color signal (Gc*) at positions where no 

signal of the given color is present; 

generating (3) an average value (avG) of the given color in dependence 
upon both said first and said second values; and 

furnishing (5) said output color signal (Go) in dependence upon both said 
10 intermediate color signal (Gc') and said average value (avG). 

2. A method as claimed in claim 1, wherein said furnishing step (5) depends 
on a presence of contours in said input signal (RGBin). 

3. A device for interpolating an output color signal (Go) of a given color in 
dependence on an input signal (RGBin) having first values of the given color which are 

15 influenced by a first other color and second values of the given color which are influenced by 
a second other color, the device comprising: 

means for interpolating (1) an intermediate color signal (Gc') at positions 
where no signal of the given color is present; 

means for generating (3) an average value (avG) of the given color in 
20 dependence upon both said first and said second values; and 

means for furnishing (5) said output color signal (Go) in dependence upon 
both said intermediate color signal (Gc') and said average value (avG). 

4. A camera, comprising: 

a sensor (S) for furnishing an input signal (RGBin) having first values of a 
25 given color which are influenced by a first other color and second values of the given color 
which are influenced by a second other color; and 
a device as defined in claim 3. 
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